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Selection or Self-Selection? How Rigorous are the Selection Criteria 
in Education Administration Programs? 



Theodore Creighton, Ed. D. 
Sam Houston State University 

Gary Jones, Ed. D. 

Idaho State University 



Abstract 

The present study is one of the first large-scale investigations of existing selection criteria and 
procedures in university principal preparation programs. Over 450 university master's degree 
and principal certification programs in education administration were reviewed. The results are 
discouraging. Except for a few bright spots revealing rigorous and quality selection procedures , 
the majority of university preparation programs utilize criteria neither remarkable nor 
characterized by validity and reliability. In a field (education administration) where great energy 
is expended to develop u best practices ” and “ innovative strategies" for schools and students , it is 
noteworthy that assumptions behind current selection criteria for principals and other 
administrators are not only underdeveloped, but basically unchallenged. If one wishes to improve 
practice in P-12 schools, an obvious place to start is found within the development of future 
principals and other administrators at universities. A plethora of opportunities exist at 
Institutions of Higher Education (IHEs) for study of innovative selection procedures within 
existing principal preparation programs. The authors of this study believe that one of the most 
important steps a university can take to positively impact education is to better identify 
appropriate candidates for these programs. 



Introduction 

Effective education leadership programs consist both of program experiences and 
the quality of entering candidates. The selection of candidates is fully as critical as the 
preparation program itself. After 40 years of continued alerts (AASA, 1960; Creighton, 
2001; Farqchar & Piele, 1972; Murphy, 1999; NPBEA, 1989; Pitner, 1990), the field of 
education administration still has serious problems in the ways candidates are selected 
into education leadership programs. 

A review of the empirical research related to candidate selection in education 
administration programs reveals: (a) a scarcity of comprehensive studies and (b) the 
utilization of excessively small sample sizes (e.g., Murphy, 1999; Pounder & Young, 
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1996). With more than 500 existing universities and colleges preparing principals (and 
others appearing overnight via distance learning), a dire need exists for a meta-analysis of 
selection criteria used. 

The National Policy Board for Educational Administration (NPBEA) published 
Improving the Preparation of School Administrators (1989), a bulletin emphasizing the 
need to strengthen the procedures used to identify and select students into education 
administration programs. They recommended that: 

...entrance standards to administrator preparation programs be 
dramatically raised to ensure that all candidates possess strong 
analytical ability, high administrative potential, and demonstrated 
success in teaching. (NPBEA, 1989, p.5) 

There is some evidence that a more proactive stance to the selection of potential 
school leaders in the nation’s university preparation programs has taken place over the 
last decade. Murphy (1999) states that selective measures have expanded beyond the 
traditional to include assessment center activities and interviews. Looking closely at 
Murphy’s research, one can identify two potential issues related to validity and reliability. 
First, not only were the sample sizes relatively small (N = 44), but data were collected 
from department chairs from a rather elite group of institutions. University programs 
reviewed were members of the University Council of Educational Administration 
(UCEA). At the time of Murphy’s study, UCEA membership included 54 universities. 
The quality of preparation program at UCEA institutions is not in question. But it is 
noteworthy to add that membership is cost prohibitive to many universities, and UCEA 
membership represents only approximately one-tenth of university preparation programs. 
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Research on selection criteria ought to reflect and include the approximate 500 
universities who prepare a significantly higher number of principal candidates. To 
repeat, the quality of program at UCEA institutions or non-UCEA schools is not in 
question here. The issue relates to the small, highly selective group of institutions 
studied. 

The second concern of Murphy’s findings relates to the question of whether or not 
department chairs are the most accurate source of valid and reliable data. Creighton and 
Jones (2001) found conflicting information in regards to selection criteria depending on 
the person supplying such data. Department chairs were found to supply more positive 
responses in regards to the rigor and quality of candidate selection. Reviewing university 
catalogs, and interviewing program directors, college deans, secretaries, and individual 
university professors, revealed some discrepancy with department chairs’ descriptions of 
existing university selection procedures and practices. At several institutions “special 
circumstances” allowing inclusion of otherwise unqualified individuals into programs 
were commonly applied throughout the selection process. 

Upon a more thorough inspection of a larger sample (N = 450) however, the 
majority of universities still rely primarily on Graduate Record Examination scores, 
letters of recommendation, and grade point averages (Creighton & Jones, 2001). 

Evidence of the use of strategies focused on “analytical ability, high administrative 
potential, and demonstrated success in teaching” was minimal at best. 

University preparation programs are under pressure to take in adequate numbers 
of candidates to justify the program’s costs and existence (Sarason, 1999). This pressure 
results in admitting individuals of borderline quality, with a continued dependence on 
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traditional selection criteria such as Graduate Record Examination scores, grade point 
average, and letters of recommendation. With the projection of approximately 50 per 
cent of current school administrators leaving the nation’s public schools in the next 
decade (Harris, Arnold, Lowery, & Crocker, 2000), it is more than crucial that 
universities attract and select high quality candidates to school administration preparation 
programs. This study warns that no aspect of a university’s principal preparation 
program is more damaging than a ubiquitous reputation for being a refuge for mediocre 
candidates. 

The Problem 

The field of education administration has long been criticized for the ways in 
which men and women are prepared for school leadership positions. In 1960, the 
American Association of School Administrators (AASA) characterized the preparation of 
superintendents and principals as a “dismal montage” (Murphy, 1999, p. 84). Later 
Farquhar and Piele (1972) described university-based preparation programs as 
“dysfunctional structural incrementalism” (p. 17). As recently as 1990, Pitner discussed 
the “zombie programs” (p. 131) in educational administration. 

The problem relates to how we select our prospective school administrators for 
our university preparation programs. The most common selection strategies accept future 
administrators into programs under three criteria: (a) Graduate Record Examination 
(GRE) scores, (b) grade point average, and (c) letters of recommendation, with little 
evidence of consistent standards of acceptance. Recent research has identified the criteria 
weighted most heavily in the selection of candidates as the GRE score (Norton, 1994). 
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The present study finds the same practices in place today, with little change in selection 
criteria. 

A secondary problem is whether or not current admission standards are 
sufficiently high to attract the “brightest and most capable candidates” to our programs 
(Norton, 1994, p. 41). Data published in the Graduate Record Examination (GRE) Guide 
(2001) reveal that students entering education administration preparation programs from 
1996 to 1999 attained GRE scores ranking third from the bottom when compared with 
forty-one graduate fields. Compared with seven specific education majors, education 
administration ranked second from the bottom in verbal reasoning, third from the bottom 
in quantitative reasoning, and second from the bottom in analytical reasoning. 

The present use of GRE scores, undergraduate grade point averages, and letters of 
recommendation for the selection of candidates for school administration in isolation are 
not in question - they have their place - but when they are the sole basis for selection, 
they are and have been found to be only partially effective (Sarason, 1999). Although 
various selection criteria are used, the dominant one is Graduate Record Examination 
scores; only limited attention is given to factors associated directly with administrative 
potential (Creighton & Jones, 2001). 

The lack of rigorous selection procedures has several potentially negative effects: 

1. Weak selection processes lower the quality of instruction offered, since 
courses and instruction are often geared to the background and intelligence of 
the students; 

2. Easy entry diminishes the status of education administration programs in the 
eyes of the public; 
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3. The candidates themselves realize that anyone can get the credential if he or 
she keeps paying for credits; (Cooper & Boyd, 1987, p. 14) and 

4. Low standards of admissions permit and encourage enrollment of candidates 
interested only in a master’s degree in education with little intent of 
vigorously pursuing an administration position. (Creighton & Jones, 2001) 

Increasing the level of selection will likely result in a higher quality of 
administrator prepared by IHEs, thus providing better principals and superintendents for 
the nation’s schools. Many studies point to the importance of quality leaders (Fullan, 
1994; Sarason, 1995; Sergiovani, 1996; Schlecty, 1997), and support exists for the stance 
that no amount of significant education reform or restructuring will occur without strong 
effective school leadership. The prediction that as many as 50 percent of school 
administrators in the nation will leave their positions in the next decade due to retirement 
and career change translates to the potential of replacing approximately half of the 
education leaders in our country in a relatively short period of time. Emphasizing that 
strong effective leaders throughout the nation have a direct effect on student achievement 
and organizational change, the impact on and potential for implementing major education 
reform is immense. If such a number of school administrators are replaced in the next 
decade, a window of opportunity exists to radically improve the quality of education. 
Improving the selection of school administrators must become a higher priority in all 
university preparation programs. How will the greater education community respond to 
this need for numbers? Will IHEs address the supply and demand with current selection 
processes , or will they focus on the opportunity to improve quality and effectiveness in 
education? If the latter, education leaders must be certain that university selection 
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procedures are rigorous, effective, and cost efficient. A specific fear is the possibility 
that universities might even lower standards of admission to accommodate the growing 
need for candidates allowing the issue of supply and demand to undercut the quality of 
prospective school leaders admitted to education administration programs. 

Purpose of the Study 

It is not the purpose of this paper to suggest the elimination or a de-emphasizing 
of the commonly used methods of selection such as GRE scores, GPA, and letters of 
recommendation. Attention is drawn to the results of the present study revealing the use 
of these methods as the sole basis for selection to the exclusion of somewhat more 
reliable measures - assessment center activities, situational interviews, teaching 
experience, and more specifically, the audition. The authors suggest a potentially more 
effective approach to candidate selection, including a wide variety of traditional and non- 
traditional practices. 

The purpose of the study was to: (a) determine the extent to which rigorous 
selection processes exist and (b) identify exemplary models resulting in the selection of 
candidates possessing strong qualifications and characteristics of school leadership. 

The specific research question relates to whether admission standards are 
sufficiently high to attract and identify candidates of diverse race, ethnicity, and gender 
who are likely to succeed in this challenging field. 

Methodology 

To identify colleges and universities with education administration programs, 
several data sources were consulted. The Eighteenth Edition of the Educational 
Administration Directory (Lane, 2000) identified approximately 350 university programs 
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preparing school principals. The researchers identified an additional 100 education 
administration university programs through two Internet web sites: (1) 
http://www.liuxueguide.com and (2) http://www.gradschools.com . The admissions and 
selection procedures of universities in all geographic regions were analyzed, including: 

(a) Western Plains, (b) Mid-eastern, (c) Mid- Atlantic, (d) North-eastern, (e) South- 
eastern, (f) Mid-south, and (g) Western. 

Individual university web sites were visited accessing graduate school catalogs 
(admissions criteria) and any additional specific requirements from individual 
departments of education administration. Of the 450 universities analyzed, a 10% 
purposeful sample (N=45) was selected for follow-up personal communication. Deans of 
graduate schools, department chairs, and other department personnel were asked to verify 
data collected from graduate school catalogs and web sites. 

Data collected included the following variables: 

1. Name of university 

2. Bachelors degree required 

3. Undergraduate GPA 

4. Graduate GPA 

5. GRE and/or MAT requirement 

6. Combination admissions formulas 

7. Letters of recommendation 

8. Writing sample 

9. Provisional admission 

10. Teaching experience (years) 
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11. Interview 

12. Assessment center activities 

The study utilized a descriptive research method, collecting descriptions of 
education administration programs in regards to selection and admission criteria. In data 
analysis, measures of central tendency (mean), measures of variability (standard 
deviation), and percentages were reported. It was not the intent of the researchers to 
imply cause and effect, as in a causal-comparative design, where relationships between 
variables are studied. The researchers’ desire is to provide statistical information of 
interest to educators and policy makers in administrative preparation programs. 

Certainly, a next step might include further research of the nature of causal-comparative 
or correlation studies. 

Results 

Finding 1 

The most frequent combination of selection criteria were: 

1. Bachelors degree from an accredited college or university 

2. Undergraduate grade point average 

3. Graduate Record Examination scores (Verbal and Quantitative Reasoning) 

4. Letters of recommendation (2-3) 

5. Goals/Purpose statement 

Discussion 

Within these criteria, the researchers found a wide variance existing in each. For 
example, though 100% of the universities required a bachelor’s degree from an 
accredited institution, only 3% (15) required the degree to be in education or one strong 
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in the liberal arts. One might suspect a degree in education would be helpful in preparing 
school administrators, but apparently this is not presently an issue with graduate schools 
and departments of education administration. This is not to suggest that candidates with 
other than education degrees should be eliminated from the pool, but departments might 
benefit from considering the education major as part of the selection criteria. 

More discrepancy exists in regards to required undergraduate grade point 
averages. Table 1 displays the distribution of universities and departments and their 
accompanying requirements: 

Table 1 

Undergraduate Grade Point Average Requirements 



Undergraduate GPA 


Number of Universities 


Percent of Total N (450) 


3.0 or higher 


194 


43% 


2.75 or higher 


124 


28% 


2.5 or higher 


132 


29% 



In addition, all universities listing a requirement for graduate grade point average stated a 
requirement of 3.0 or higher. 

The most commonly used criterion with regards to the Graduate Record 
Examination (GRE) is a combined cut-off verbal and quantitative score of 1000, though 
many listed cut-off scores as low as 700. Approximately 52% (234) of the universities 
state specific cut-off requirements in either the GRE or Millers Analogy Test (MAT). 

The authors discuss this subject in greater detail later in this paper. Approximately 2% 
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(9) of the universities reviewed use a combination formula rather than a specific cut-off 
for the GRE. For example, one university uses the formula: 200 X GPA + GRE (V and 
Q) = 1500, allowing for a GPA as low as 2.5, accompanied by a V and Q total of 1000. 
The other 8 universities use a similar formula, with a slightly different beginning number, 
one of which requires the Verbal and Analytical components in the formula. 

Only two universities require the analytical reasoning score in their selection 
criteria. When interviewed, these higher education administrators reported their belief in 
a relationship between analytical reasoning skills and leadership potential. Four 
universities allow the option of using the quantitative or analytical score in the selection 
process. The authors recommend strongly the need for additional research and validity 
and reliability studies to further investigate the possible relationship between analytical 
scores and leadership potential, especially in the light of the NPBEA recommendation 
(1989) that candidates “possess strong analytical ability.” Of special note is the fact that 
ETS has recently announced the addition of a written analytical reasoning component to 
the GRE examination in October 2002 (Houston Chronicle, July 2, 2001, p.4). 

The most prevalent criteria used currently in the selection of candidates in 
education administration preparation programs is the use of the Graduate Record 
Examination score. Though many universities use the GRE score in combination with 
other factors such as letters of recommendation, undergraduate grade point average, and 
writing samples, still more than half (52%) state specific cut-off points required for 
admission (e.g., V + Q = 1000). Evidence exists indicating some graduate schools and 
departments of education administration initiate the cut-off to expedite the admissions 
process due to tight schedules and insufficient staffing (Educational Testing Service, 
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1995). This is especially noteworthy due to the fact that both ETS and the GRE program 
specifically warn against setting such arbitrary cut-off points. 

Why Not 1000? Applicants whose scores fall below an arbitrary cut-off often are 
given little or no consideration in the admissions process. Figure 1, based on GRE 
examinees from the 1993-94 testing year (N=417 K) and published by the ETS illustrates 
why it is inadvisable to routinely disregard applicants whose combined verbal and 
quantitative scores fall below 1000. 

Figure 1 presents the percent of ethnic subgroup members in particular who are 
disadvantaged by a combined verbal and quantitative cut-off of 1000. As shown, 86% of 
African American examinees have combined scores below 1000; 76% of Puerto Ricans 
score below 1000; and 69% of Mexican Americans and 61% of American Indian and 



Hispanic examinees score below 1000. 
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Figure 1. Combined GRE verbal and quantitative scores for selected groups. 
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In only three subgroups - those who describe themselves as White, Asian 
American, or Other - does a cut-off point of 1000 discount less than half of the 
population. Approximately 45% of White examinees, 43% of Other examinees, and 38% 
of Asian American examinees fall below the combined verbal and quantitative score of 
1000 . 

Figure 2 shows a combined score performance for subgroups of citizenship and 
gender. Data on citizenship reveal a larger percentage of U.S. citizens (48%) than non- 
U.S. citizens (39%) fall below the 1000 cut-off. Concerning gender, 55% of females had 
combined scores of less than 1000, while only 35% of males did. 

Combined GRE Verbal & Quant. Scores 
for Selected Groups 

1300 
1200 
1100 
1000 
900 
800 
700 
600 

Total US Citizen Non US Citizen Male Female 

|=| Score at 20th %ile for This Group 1=3 Score at 80th %ile for This Group 
« Score at 50th %ile for This Group N=41 7K 

Figure 2. Combined GRE verbal and quantitative scores for selected groups. 




Clearly, large numbers of prospective graduate students in a variety of subgroups 
are adversely affected by a cut-off score policy. Combining any two or three scores (V, 



14 



Q, A) presents a special problem: Candidates may have acceptable, even commendable, 
scores in one area considered important to success in education leadership (such as verbal 
reasoning), but their score in the other area (such as quantitative reasoning), that may be 
less related to success in education administration, may be keeping them out of 
consideration. For example, a candidate could potentially have a verbal score of 800 
(perfect), but a score of 150 in quantitative, and not meet the cut-off criteria. 
Recommendations 

The authors are not suggesting the elimination of the GRE from the criteria used 
to select education leadership candidates. GRE scores are certainly an important measure 
for consideration in selecting prospective school leaders. The danger lies in their 
inappropriate use: either as a sole criteria or in conjunction with a set cut-off score. It is 
recommended, however, that further research be conducted investigating any relationship 
between leadership potential and results of the analytical section of the GRE. The 
authors also recommend discontinuance of the cut-off score policy of any kind, with or 
without combination formulas. Further, scores from the different measures should not be 
compared, because each measure is scaled separately. 

Summing GRE verbal and quantitative scores, or any other summing variation, 
hides the differences between the applicants. Summing the scores, then arbitrarily 
applying a minimum combined score may eliminate qualified candidates from the 
applicant pool. GRE tests provide measures of certain types of developed abilities and 
achievement, reflecting educational and cultural experience over a long period. Special 
care is required in interpreting the GRE score of students who may have had educational 
and cultural experiences somewhat different from those of the traditional majority. 
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Departments of education administration and graduate schools must look closely 
at the three individual areas of the GRE (i.e., verbal, quantitative, and analytical) and 
consider “weighting” areas felt to be more related to success in education leadership 
programs. It is reasonable and expected that weighting would vary (and rightfully so) 
from community to community and the philosophy of independent faculty. Along with 
the GRE program, the authors issue a caution: Weighting a particular component, given a 
program’s emphasis on particular skills is only appropriate when based upon empirical 
evidence, such as a validity study to establish an appropriate weight for each measure 
(GRE, 2001). 

Finding 2 

Data published in the Graduate Record Examination (GRE) Guide reveal that 
students entering education administration preparation programs from 1996 to 1999 
attained GRE scores ranking third from the bottom when compared with forty-one 
graduate fields. Compared with seven specific education majors, education 
administration ranked second from the bottom in verbal reasoning, third from the bottom 
in quantitative reasoning, and second from the bottom in analytical reasoning. 

The report begins by showing the data in relationship to “intended broad graduate 
major fields.” These data are based on the performance of seniors and non-enrolled 
college graduates who tested between October 1, 1996, and September 30, 1999. Please 
note that the data represents “intended majors,” meaning examinees responding to the 
department code question. Examinees responding “Undecided” are not included in these 
data. Table 2 displays the results of candidates for education programs compared to six 
other broad major fields (most Business majors take the GMAT, resulting in a low N). 
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Table 2 

GRE Percentage Distribution of Scores Within Broad Major Fields 







Verbal 




Quantitative 


Analytical 




N 


Mean 


S.D. 


N 


Mean 


S.D. 


N 


Mean 


S.D. 


Life Science 


116,620 


463 


95 


116,598 


555 


116 


116,487 


560 


120 


Physical Science 


39,760 


494 


118 


39,760 


675 


105 


39,750 


613 


123 


Engineering 


46,823 


470 


114 


46,824 


703 


83 


46,755 


603 


120 


Social Science 


86,709 


482 


105 


86,691 


535 


123 


86,632 


555 


125 


Humanities 


43,287 


538 


111 


43,258 


541 


120 


43,255 


577 


123 


Education 


36,606 


447 


94 


36,603 


511 


118 


36,560 


534 


123 


Business 


6,788 


440 


102 


6,788 


544 


143 


6,781 


530 


135 



Table 3 displays the results of candidates in education administration programs 
compared to all others in the broad major field of education. 

Table 3 

GRE Percentage Distribution of Scores In Education and Education Administration 







Verbal 






Quantitative 






Analytical 






N 


Mean 


S.D. 


N 


Mean 


S.D. 


N 


Mean 


S.D. 


Education 


36,606 


447 


94 


36,603 


511 


118 


36,560 


534 


123 


Education Admin. 


2,655 


425 


87 


2,655 


496 


119 


2,655 


513 


127 



Disaggregating the data further, Table 4 displays the results of candidates for 
education administration programs compared to other specific majors in education. 
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Table 4 

GRE Percentage Distribution of Scores In Education Administration and Education 



Majors 




Verbal 




Quantitative 


Analytical 




N 


Mean 


S.D. 


N 


Mean 


S.D. 


N 


Mean 


S.D. 


Secondary Ed. 


5,440 


486 


96 


5,440 


562 


119 


5,440 


577 


119 


C and I 


1,960 


466 


98 


1.960 


534 


120 


1,960 


554 


126 


School Psch. 


1,915 


454 


84 


1,915 


511 


107 


1,915 


544 


113 


Ed Psyc. 


1,117 


451 


95 


1,117 


510 


118 


1,117 


534 


119 


Elementary Ed. 


5,820 


442 


88 


5,820 


505 


108 


5,820 


534 


117 


Special Ed. 


2,320 


432 


87 


2,320 


484 


106 


2,320 


516 


119 


Education Admin. 


2,655 


425 


87 


2,655 


496 


119 


2,665 


513 


127 


Physical Ed. 


1,744 


400 


76 


1,744 


490 


108 


1,744 


504 


116 



Discussion 

Though not convinced of a direct relationship between GRE scores and leadership 
potential, the authors do think there is cause for concern in the data reported above. 
Certainly, other broad fields such as the sciences and engineering perhaps attract 
candidates with higher scores in quantitative reasoning than would education 
administration. However, with some evidence of a relationship between effective 
leadership and verbal skills such as used in oral and written communication (Coleman, 
Copeland, & Adams, 2001), one questions the revelation that education administration 
verbal reasoning scores are ranked next to last when compared to candidates in all other 
broad graduate fields. Further concern relates to the low scores of education 
administration candidates even in relationship to other education majors. When 
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compared to other education majors, education administration scores ranked second from 
the bottom in verbal reasoning, only above physical education majors. Education 
administration scores ranked third from the bottom in quantitative reasoning, ahead of 
special education and physical education majors. Education administration scores ranked 
second from the bottom in analytical reasoning, ahead of only physical education majors. 

An additional concern relates to the lower scores in education as a broad field. 
This certainly has implications for teacher education as well as education administration. 
Education as a broad major field falls significantly below all other broad fields with the 
exception of business in the areas of verbal and analytical reasoning (note that the 
majority of business majors take the GMAT exam). The authors emphasize similar 
concern in this regard, due to the fact that we are also faced with a serious shortage of 
teachers across the nation. The same question exists: Will we sacrifice quality for 
quantity sustaining the status quo, or more serious, lower standards of admission and 
selection as we strive to fill the large number of vacancies existing in teacher education 
AND school administration. 

Recommendations 

Evidence exists indicating a somewhat positive correlation between GRE scores 
and success in graduate school (ETS, 2001). However, more research is in order to 
investigate the possible relationship between higher scores in particular components of 
the GRE (e.g., verbal and/or analytical) and leadership potential. Perhaps the problem is 
more in the area of recruitment than selection. University preparation programs should 
initiate adequate resources for recruiting candidates of high quality and possessing 
characteristics of effective leadership. For example, a stronger relationship between 
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university preparation programs and practicing school administrators would help identify 
candidates with leadership potential and/or experience. 

The traditional practice of requiring letters of recommendation might be replaced 
with agreements from practicing administrators to “sponsor” or “support” candidates in 
education administration programs. Taking such commitments to sponsor, mentor, and 
support in addition to respectable GRE scores all seem to be a more logical step in the 
selection process. 

Graduate Record Examination scores are a valuable component of the graduate 
application package in education administration preparation programs. We emphasize 
the importance of combining this information with other sources to provide departments 
of education administration with important information about the skills and abilities of 
applicants. 

Finding 3 

Only 27 (6%) of the university preparation programs studied require personal 
interviews as part of the selection process. An additional 9 (2%) include a “potential” 
interview, decided after a review of other application material. The remaining 
universities (414) require no form of personal contact with candidates preceding 
acceptance (or rejection). Only 1 university in the study requires assessment center 
activities of candidates before admission (i.e., oral and written performances based on 
selected scenarios and situations). 

Discussion 

Existing research suggests that selection procedures and practices are often 
unsystematic and unstructured (Pounder & Young, 1996) and have weak predictive 
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power. Even universities with rigorous and highly structured criteria often find they are 
not able to predict leadership potential to the degree they would prefer, unless their 
selection criteria includes “significant behavior-based measures” (p. 300). Pounder 
(1989) suggests that typical screening criteria used in selection systems are “proxy 
measures” (p. 300) for the desired leadership behavior. 

The use of the interview has increased slightly over the last decade as a more 
reliable strategy in identifying potential school administrators. Though its use in the 
selection process in university preparation programs remains relatively small 
(approximately 6%), the interview is not as expensive or labor intensive as the 
assessment center activity. It is encouraging to discover the emergence of the situational 
interview, a strategy that immerses the candidate into a real-life environment. 

Research has shown the interview to contain several weaknesses. Selection 
decisions based upon the interview reveal disproportionate rates of selection between 
minority and majority members. Decisions tend to be made in the first few minutes of 
the interview with the remainder of the interview being used to validate or justify the 
original decision. 

In many cases, questions are not job related , resulting in the potential for racial 
and gender stereotypes throughout the interview process. In addition, evidence reveals a 
tendency to give negative information more weight than positive information. For the 
interview process to be effective, interviewers must: (1) avoid asking questions unrelated 
to the job, (2) avoid making quick decisions about the applicant, (3) avoid stereotyping 
applicants, (4) avoid giving too much weight to a few characteristics, and (5) maintain 
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consistency in the questions asked. Because of these confounding variables, little 
evidence of the validity exists of the interview used as a selection procedure. 

Though the interview is considered by many as the most important administrator 
screening tool (Baltzell & Dentler, 1983; Bryant, 1978; Pounder & Young, 1996; Schmitt 
& Cohen, 1990), candidate interviews are often unstructured, informal, and inconsistent. 
Even when highly structured guidelines are followed, it is difficult to predict future job 
performance, unless the selection system focuses on behavior-based measures (e.g., 
situational interviews). 

Recommendations 

Several have suggested that leadership must be viewed more as a performing art 
than a set of skills, competencies, and knowledge (Sarason, 1999; Vaill, 1989). Graduate 
schools in the performing arts certainly require traditional criteria such as previous 
grades, test scores, and interviews. But no school accepts a student without an audition - 
they want to see how the student performs. Auditions are not interviews. They are 
samples of behavior displayed under real-life situations. IHEs cannot afford to admit 
individuals into preparation programs without some form of audition that offers a basis 
for assessing how they will interact with children, parents, and teachers in situations 
calling for inventiveness, spontaneity, and sensitivity (Sarason, 1999). 

Interviews (and to some extent assessment center activities) usually focus on a 
predetermined set of questions or hypothetical situations and only allow the candidate to 
tell how and why or what he or she would do to handle the situation. Auditions, on the 
other hand, immerse the candidate into a real-life environment of education 
administration and require the candidate to actually demonstrate (perform) a behavior. 
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What a person says he or she will do is far from correlated with what a person actually 
does when confronted with the situation (Sarason, 1999). As in the performing arts, 
auditions used in administrative recruitment and selection need not require a perfect 
performance - we realize we are observing untrained individuals. But does the candidate 
display the “qualities” necessary or the “potential” for performing effectively with 
children, parents, and teachers? 

Auditions are not without disadvantage or weakness. Similar to assessment center 
activities, the audition is time-consuming and labor intensive, especially if required of 
every candidate to education administration programs. As with the interviews and 
assessment center activities, the audition sets the stage for potentially overly subjective 
judgment and decisions by university personnel. Program leaders must strive for 
consistent evaluations and objective decisions based on quality of performance and 
display of leadership potential. 

Departments of education administration might consider the use of an emerging 
practice (Creighton & Jones, 2001) of weighting formulas, allowing for all selection 
criteria to be considered but giving advantage to criteria requiring a display of behavior- 
based performance. Such a formula might look as shown in Table 5: 
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Table 5 

Weighting of Selection Criteria 



Selection Criteria 


Weight Given to Decision 


GRE Scores 


10% 


GPA 


10% 


Writing Samples 


5 % 


Letters of Support/Sponsor 


10% 


Interview 


10 % (20 % if situational ) 


Assessment center Activity 


20% 


Audition 


25 % 



Auditions provide an opportunity to observe the candidate in a realistic school 
environment. For example, auditions can focus on: (1) an unscheduled meeting with a 
hostile parent, (2) talking with a student contemplating suicide, (3) taking a phone call 
from an irate parent, or (4) presenting a report of declining standardized test scores to the 
board of education. A more specific example of auditions is displayed in Appendix A. 
Finding 4 

Approximately 180 universities (40%) studied require a teaching credential or 
teaching experience as a requirement to admission. A majority of 270 universities (60%) 
allow a candidate to complete a Masters Degree in Education Administration without 
satisfying the teaching requirement for state certification of principal or superintendent. 
In other words, the candidate can complete the degree but lack the requirement of 
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teaching experience, which is two-to-three years in most states. The university grants the 
degree, but not verification of meeting the requirements for state certification. 

Discussion 

This finding is truly an alarming one, indeed. The prevailing question is: What 
knowledge base or experience does this candidate bring to such courses as school law, 
school finance, supervision of instruction, school personnel, the principalship, and 
dealing with students of special needs? This candidate lacks first-hand knowledge and 
understanding of the school setting, students, teachers, administrators, and instruction. In 
these cases, why would we assume learning acquired in the university program can be 
“put on hold” while the candidate completes the required two or three years of teaching 
experience in the classroom? This situation again, draws attention to the fact that 
departments of education administration and its professors likely have to “adjust” or 
“water-down” the curriculum to fit the needs of students who lack any classroom or other 
school site experience. 

Universities permitting candidates to complete education administration 
preparation in this manner not only are doing a disservice to the candidates themselves, 
but also a disservice to the teachers, students, and community members in the schools 
these aspiring principals will someday attempt to lead. 

Some universities argue that if the candidates are currently in their first or second 
year of teaching, candidates will satisfy the state certification requirement by the time 
they graduate. This argument perhaps draws attention to the university’s willingness to 
sacrifice quality of candidate in the name of enrollment quotas and cash flow. In 
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addition, this situation gives support to the authors’ contention that the quality and rigor 
of instruction seeks the level of experience and intelligence of its candidates. 

Recommendations 

This practice contributes to the growing problem of placing excessive numbers of 
candidates with education administration degrees in non-administrative positions. Many 
remain in the classroom as classroom teachers. This result in itself - or in isolation - is 
not the issue. The concern relates to the amount of time between the granting of a degree 
and the actual certification of the candidate. If several years go by, the candidate brings 
outdated knowledge and skills (or even the lack of same caused by the time factor) to the 
leadership position. 

It is encouraging to discover that some states are addressing this potential 
problem. California, for example, requires three years of teaching experience to enter a 
master’s program in education administration. Upon completing the masters in education 
administration, the candidate is granted a provisional certification, and allowing three 
years to secure a position in school administration. If an administrative position is not 
secured in that time, the candidate is denied professional certification and required to 
return to the university for updated and current professional coursework. 

The authors strongly recommend that the practice of allowing candidates without 
the appropriate teaching experience to enter education administration programs be 
discontinued. Until further studies and research indicate otherwise, the profession 
believes a relationship exists between effective school leadership and a limited amount of 
classroom teaching experience or other school-site experience. 
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Finding 5 

Eighty percent of the universities (360) studied provide for a conditional 
admissions policy. Though the conditions vary, most state a conditional acceptance if 
GRE scores and/or GPA are below normal acceptance levels. In other words, if a 
candidate’s GRE scores are below acceptance, the university accepts the candidate 
conditionally, based upon a retake of the GRE, and/or the completion of 6-9 credits with 
a 3.0 or better average. 

Discussion 

The authors are not in any way suggesting this policy is inappropriate and does 
not provide marginal students a chance to succeed. The point of argument seeks to 
address the nature of the condition: The condition again is related to success in graduate 
school, rather than an indication of leadership potential. What is the relationship between 
a student maintaining a B average in the first 6-9 credits (which are usually core courses 
and rarely courses in school administration) and potential in principal preparation 
programs. Perhaps these variables may relate to success in graduate school, but evidence 
points to little relevance to effective leadership characteristics and potential. 

Habick (2001) suggests that from ETS’s perspective, there is no reason for a 
candidate to take the examination the second time, if he or she prepared properly. 
Research conducted by ETS on test results of examinees repeating the GRE examination 
for a second time reveals only a slight increase in scores. The authors’ experience at five 
universities includes no evidence of candidates scoring significantly higher (usually 2-10 
points, combined) on an additional try. The practice of requiring candidates to take the 
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examination a second or third time places unnecessary stress on the candidate, along with 
additional expense, with little evidence the candidate will score any differently. 

In the authors’ experience with several university preparation programs, they have 
yet to witness a rejection of candidate at the master’s level in education administration. 
Again, the authors are not lobbying for rejection of candidates, but instead argue for high 
quality, rigorous, and fair selection procedures. Looking at any other rigorous selection 
procedures (e.g., other graduate fields), decisions are made rejecting candidates with 
questionable admissions characteristics. A repeated theme: The education 
administration profession has a reputation for being a refuge for mediocre candidates. 

Conclusions 

Effective education leadership programs consist of both quality of program and 
quality of entering candidates. In a time when we are focusing on higher standards for 
education leadership programs, reform in university preparation programs is not likely, 
and will not begin, until we insist on candidates with strong potential for school 
leadership. 

Final Thoughts 

Jones: 

When I was fourteen I lived in a small farming community. A local farmer 
persuaded me to milk his dairy herd while he took an extended Christmas vacation. I was 
supposed to milk thirty head of cows each morning and evening for a month. About the 
beginning of the second week the amount of milk production began to decline. Instead of 
sixty gallons of milk per milking I was only receiving 45-50. 1 was concerned that the 
farmer would be angry with me for decreased production, so I added water to the milk to 
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make it look like everything was fine. Even though I felt guilty, no one could see the 
difference, and I believed that my deception was unsuspected. In fact, the farmer checked 
the poundage charts from the driver upon his return and seemed fairly happy with my 
efforts. Eventually, though, the milk-fat test results (ones I didn’t know were kept) were 
reported to him. He discovered that the quality was way down, and figured out what I had 
been doing. For a short while inferior quality work was covered up, and I looked good. It 
was not until later that the results of my cover-up were discovered and I had to face the 
reality of the situation. 

The analogy, it seems to me, is clear. At IHEs a balance exists between faculty 
and students. Poor quality — whether from the educator or the student — creates an 
imbalance that leads to failure. On one hand education needs high-quality instructors and 
professionals who can deliver an educational product that is excellent. On the other hand, 
education needs top-rate students who can understand and utilize this excellent 
information. Without either of these, failure abounds. 

Creighton: 

It is interesting to note the heavy emphasis on quality of university program (e.g., 
NCATE, ISLLC, NPBEA) with an accompanying absence of emphasis on quality of 
candidates selected to education administration programs. Only in the NPBEA standards 
document currently “in progress,” is there a mention of rigorous selection criteria (e.g., 
input standards). Why is it assumed that high quality effective administrators can be 
produced by university programs without first attracting and selecting candidates with 
existing potential of school administration and leadership? And why do IHEs still utilize 
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the common non-behavior-based selection criteria that show no evidence of relationship 
with leadership skills, knowledge, or dispositions? 

Along with existing selection criteria, university programs must give thought to 
considering more behavioral-based criteria to better identify candidates with leadership 
potential. Until all involved parties seriously address the selection of candidates with 
strong analytical ability, high administrative potential, and demonstrated success in 
teaching, the education profession will continue to be a refuge for mediocre candidates. 

References 

Baltzell, D, & Dentler, R. (1983). Selecting American School Principals: Research 
Report. Cambridge. MA: Alt Publishing. 

Bryant, B.J. (1978). Employment factors superintendents use in hiring administrators for 
their school districts. Prepared for the Association for School, College, & 
University Staffing. (ERIC Document Reproduction Service No. ED 9866) 
Coleman, D.G., Copeland, D.G., & Adams, R.C. (2001). University education 

administration program development; Administrative skills vs. administrative 
standards as predictors of administrative success. In Kowalski, T.J. (Ed.), 2001 
National Council of Professors of Educational Administration Yearbook (pp. 25- 
35). Scarecrow Press. 

Cooper, B.S., & Boyd, W.L. ( 1987). The evolution of training for school administrators. 
In J. Murphy & P. Hallinger (Eds.), Approaches to Administrative Training (pp. 
3-27). Albany: SUNY Press. 

Creighton, T., & Jones, G. (2001, February). A meta-analysis of 450 university 



ERIC 



31 



30 



preparation programs and their selection strategies . Paper presented at the Annual 
American Association of School Administrators, Orlando, FL. 

Creighton, T., & Jones, G. (2001, August). Selection or self-selection? How rigorous are 
selection criteria in education administration programs ? Paper presented at the 
National Council of Professors of Educational Administration Conference, 
Houston, TX. 

Creighton, T. (1996). A construct validation of the Administrative Diagnostic Inventory. 
Unpublished doctoral dissertation, University of California Davis - California 
State University Fresno, California. 

Educational Testing Service (2001). Graduate Record Examination guide to the use of 
scores (2001). Princeton: NJ. 

Farquhar, R.H., & Piele, P.K. (1972). Preparing educational leaders: A review of recent 
literature . (ERIC/CEM state of the knowledge series, number 14; UCEA 
monograph series, number 1). Danville, EL: Interstate. 

Fullan, M. (1994). Change forces: Probing the depths of educational reform. San 
Francisco: Jossey-Bass. 

Graduate Record Examination to add new analytical section. (2001, July 2). The 
Houston Chronicle , p.4. 

Habick, T. (July 12, 2001). Phone interview with Timothy Habick, Assessment 
Specialist, Educational Testing Service. Princeton, NJ. 

Harris, S., Arnold, M, Lowery, S., & Crocker, C. (2000). A study of motivators and 
inhibitors for men and women deciding to become a principal. Education 
Leadership Review, 1 (3), 30-37. 




32 



31 



Lane, K.E. (2000). Educational Administration Directory, 17 th Edition. Redlands: CA. 

Murphy, J. (1999). Changes in preparation programs: Perceptions of department chairs. 
In J. Murphy & P.B. Forsyth (Eds.). Educational administration: A decade of 
Reform (pp. 170-191). Thousand Oaks; Corwin. 

National Policy Board for Educational Administration (1989). Improving the 

preparation of school administrators: The reform agenda . Charlottesville, VA. 

Norton, M.S. (1994). Student recruitment and selection practices in educational 
administration programs. Arizona State University. (ERIC Document 
Reproduction Service No. ED 366 087) 

Pitner, N.J. (1990, September). Reinventing school leadership (pp. 129-131) [Working 
memo prepared for the Reinventing School Leadership Conference]. Cambridge, 
MA: National Center for Educational Leadership. 

Pounder, D.G., & Young, I.P. (1996). Recruitment and selection of educational 

administrators: Prioities for today’s schools. In K. Leithwood, J. Chapment, D. 
Corson, P. Hallinger, & A. Hart (Eds.), International Handbook of Educational 
Leadership (pp. 279 - 308). Boston: Kluwer. 

Sarason, S.B. (1999). Teaching as a performing art . New York: Teachers College Press. 

Schlechty, P. (1997). Inventing better schools: An action plan for educational reform. 

San Francisco: Jossey-Bass. 

Schmitt, N., & Cohen, S.A. (1990). Criterion-related validity of the National Association 
of Secondary School Principals Assessment Center . Reston, VA: NASSP. 

Sergiovani, T. (1996). The roots of school leadership. Principal. 74 (2). 6-9. 

Vaill, P.V. (1989). Managing as a performing art. San Francisco: Jossey-Bass. 




33 



32 



APPENDIX A 

Auditioning in Education Administrative Preparation Programs: An Application 

Auditions were a required component of the principal preparation program at a 
mid-western university from Fall semester 1997 through Spring semester 2000. During 
that time, a total of 225 master’s students preparing for the principalship went through an 
audition process. Though the process was used in this case as a way to identify strengths 
and weaknesses of enrolled candidates, the intent of this paper is to suggest and 
recommend that auditions be implemented as an additional strategy for selection of 
candidates before they are accepted into preparation programs. 

As part of the course entitled "The Principalship," students were required to 
attend a full-day (Saturday) session consisting of a variety of auditioning activities. No 
individual names were used during the day; students began the day as a hypothetical 
principal and were identified only by a number (e.g., A-l, A-2, A-3, etc.). Activities were 
scheduled in several rooms and individual offices, and in a manner whereby each student 
progressed through the activities during the day. 

Evaluators and judges were practicing teachers, principals, superintendents, 
university professors, and students. Each candidate’s performance was judged and 
reviewed by at least three evaluators. The following example of audition activities used 
during 1997-2000 comes from the ERIC Clearinghouse on Educational Management’s 
Project-Based Learning Project (Bridges, 1994). 
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Audition #1. Irate Parent Phone Call 

Students answered a phone call from an angry parent at scheduled times in a location 
previously announced (e.g., private office). Evaluators performed the role of the irate 
parent making the call from another office, and accompanied by two additional 
evaluators. The parent script follows: 

You are Mr. Robert Wills and want your son transferred out of Mrs. Jones’ room. 
Your son is Joe, an 8th grader. 

You have aspirations for him to attend college and it is important that he do well in 
school and learn. 

Your son Joe is not learning anything in Mrs. Jones’ class - it is a total waste of time. 
You want your son transferred to Mrs. Johnson; Joe likes her and learned a lot from 
her when he had her last year. 

There must be other parents who feel the same way - have you heard from any of 
them? What are you going to do about it? 

Indicate you don’t care if the principal denies the request. 

Your son Joe is a special case, and the principal better reconsider. 

If he doesn’t transfer Joe, he will hear from you. You mean business. 

And you intend to talk with the Board President and Superintendent. 

The candidate’s responses are listened to on a speaker phone allowing the 
evaluators to hear the conversation. Both the irate parent and the candidate are in the 
privacy of individual offices, and the candidate is in no way identified by name. How is 
the candidate’s audition evaluated? 
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The evaluation form consists of administrative constructs and descriptors taken 
from NAESP, NASSP, and NPBEA assessment documents used in professional 
assessment center simulations. A completed assessment form used in the irate parent 
phone call audition is shown in Table 1. 

The important issue, as with auditions in the performing arts, is not a perfect 
performance necessarily, but whether or not the candidate possesses the important 
qualities required in the handling of an irate parent phone call. You will notice that 
additional comments focus on constructive criticism and allow the candidate to reflect on 
areas of improvement. The assessment obviously involves subjective decisions by the 
evaluators. These decisions, however, are based on many years of experience of 
practicing teachers, administrators, and university faculty. The point is that our profession 
is currently depending excessively on objective data (test scores and grade averages) 
when accepting candidates into principal preparation programs. Sarason (1999) argues 
that selecting only on the basis of conventional objective data is not justifiable on moral 
and educational grounds. He continues by making an analogy to the performing arts: "if 
you want to predict who will make a good actor, you have to see them act, keeping in 
mind that you are observing an amateur" (p. 99). 

Audition # 1 - Discussion 

The real strength of the audition provides an additional (and perhaps more authentic) 
measurement of one’s ability to perform an actual task from the school administrator’s 
day. The interview certainly addresses what a candidate "might do" in a particular 
situation, but the audition begins to focus on the issue of what the candidate "will actually 
do" in a real-life situation. The point here, again, is not to substitute the audition for the 
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interview but to include both in the overall assessment of potential for school leadership 
in perspective and aspiring candidates. 

We are beginning to view leadership less and less as consisting of quantifiable 
characteristics measured by objective test scores (GRE) and good grades in academic 
courses (GPA). However, no one would suggest that low GPA and GRE scores are 
characteristics of effective educational leaders. These criteria have their place but seem to 
be part of a larger picture. 

Evidence continues to mount indicating educational administration is less 
objective and "more dependent on the comings and goings of personalities," says Gary 
Wills (1994) in his best-selling book. Certain Trumpets, about the nature of leadership. If 
we desire to (and I suggest we must) measure the complex components of this personality 
in leadership situations, it seems that the audition more appropriately and accurately 
provides such a measurement. Do GRE scores and GPA allow us to consider this 
important leadership characteristic? I suggest not. At least, the audition should be 
included in the variety of selection strategies used in identifying the "brightest and most 
capable" for our educational administration preparation programs. 
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Table 1. Irate Parent Call Assessment Form 
Student Identification Number A-l 



Communication “look-fors” 


1 


2 


3 


4 


5 


Conveys ideas and opinions succinctly 






X 






Expresses clear and concise language 






X 






Checks for understanding 




X 








Uses appropriate language 






X 






Exhibits sensitivity to parent’s concerns 




X 








Renders a timely and appropriate decision 




X 








Displays appropriate listening skills 




X 









ADDITIONAL COMMENTS: 



1. Appeared in a hurry to get the parent off the phone. 

2. Be careful about putting the responsibility on the parent to check these issues out - 
that’s why he’s calling you. 

3. Be more explicit about what steps you will take. 

4. Strive to show empathy to the parent and at the same time be supportive of the 
teacher. 

Note: The administrator constructs and descriptors listed above are to be used as "look-fors" as the 
audition activity unfolds. Additional comments will help with the assessment of the individual. 
Observation should be rated on a scale of 1-5, with 1 indicating little evidence and 5 representing strong 
evidence. 
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